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The  purpose  of  this  project  was  to  organize  and  conduct  a 
symposium  on  the  recognition  and  classification  of  complex 
(non-speech)  auditory  and  visual  signals.  The  primary  objective 
of  the  symposium  was  to  determine  the  current  state  of  knowledge 
regarding  the  theory  of  complex  signal  recognition  and 
classif ication,  empirical  tests  of  these  theories,  and 
significant  implications  of  this  work  for  applied  recognition  and 
classification  systems.  The  meetings  also  served  to  define  some 
of  the  problem  areas  in  theory  and  research,  and  to  predict 
promising  directions  for  future  work. 

The  symposium  was  held  at  Bolt  Beranek  and  Newman  Inc.  in 
Cambridge,  Massachusetts  between  June  28  and  June  30  of  1978. 
The  co-organizers  of  the  symposium  were  David  J.  Getty  of  Bolt 
Beranek  and  Newman  Inc.  and  James  H.  Howard,  Jr.  of  The  Catholic 
University  of  America.  Thirteen  invited  speakers  participated  in 
the  meetings,  covering  the  fields  of  human  auditory  perception, 
human  visual  perception,  theoretical  and  machine  approaches  to 
pattern  recognition  and  classification,  and  multidimensional 
scaling  theory.  The  program  of  the  symposium  is  given  in 
Appendix  A;  a  list  of  the  attendees  is  given  in  Appendix  B. 

Prior  to  the  symposium  we  reached  an  agreement  with  Lawrence 
Erlbaum  Associates,  Inc.  that  they  would  publish  the  papers 
presented  at  the  meeting  in  a  hardcover  volume,  tentatively 
entitled  Auditory  and  Visual  Pattern  Recognition.  In  the  -nonths 


2 


Report  No.  4376 


Bolt  Beranek  and  Newman  Inc. 


following  the  meeting,  we  ultimately  received  manuscripts  from 
twelve  of  the  thirteen  conference  speakers.  Each  manuscript  was 
reviewed  and  edited  by  Getty  and  Howard,  as  editors  of  the 
volume,  and,  if  required,  revised  by  its  author (s).  The  final 
revisions  of  these  manuscripts  have  now  been  sent  to  L.  Erlbaum 
Associates  for  typesetting.  Our  expectation  is  that  the  volume 
will  be  released  early  in  the  fall  of  1980. 

The  contents  of  the  volume  are  summarized  by  the  following 
set  of  chapter  abstracts: 


SECTION  I:  PERCEPTION  OF  COMPLEX  AUDITORY  PATTERNS 


CHAPTER  1:  Pitch  Perception:  An  Example  of  Auditory 
Pattern  Recognition 

Frederick  L.  Wightman 

Northwestern  University 

Modern  research  in  psychoacoustics  has  provided  an 
impressive  body  of  data  on  the  auditory  system's  response  to 
simple  stimuli  such  as  pure  tones  and  clicks.  Of  course,  it  has 
always  been  clear  that  our  eventual  goal  is  to  understand  the 
processing  of  more  complex  stimuli,  such  as  speech  and  music. 
One  important  feature  of  many  complex  stimuli  is  that  they  evoke 
the  perception  of  pitch.  Neither  the  exact  physical 
characteristics  of  stimuli  which  lead  to  this  percept,  nor  the 
nature  of  the  auditory  processing  which  mediates  it  are  known  at 
present.  The  greatest  challange  for  theories  of  pitch  perception 
is  to  explain  the  invariance  of  the  percept.  Many  sounds  which 
have  very  different  physical  features  (both  temporal  and 
spectral)  have  the  same  pitch;  there  is  clearly  no  simple 
physical  correlate  of  the  pitch  percept.  Modern  theories  treat 
pitch  perception  as  a  kind  of  auditory  pattern-recognition,  in 
which  the  central  auditory  system  extracts  pitch  by  some  set  of 
operation  on  "patterns"  of  neural  activity  supplied  by  the 
periphery.  While  the  theories  differ  in  many  ways,  all  are 
remarkably  successful  in  accounting  for  the  empirical  data  and 
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there  is  little  reason  to  favor  one  over  another.  Recent 
research  from  our  laboratory  on  pitch  perception  in 
hearing-impaired  listeners  may  provide  critical  tests  of  certain 
key  features  of  the  current  theories.  Our  principal  concern  at 
present  is  the  relative  importance  of  spectral  and  temporal  cues. 
It  is  our  hope  that  these  experiments  will  be  revealing,  about 
pitch  perception  in  particular,  and  about  auditory  function  in 
general. 


CHAPTER  2:  Perception  of  Sound  Signals  at  Low 

Signal-to-Noise  Ratios 

Reinier  Plomp 

Institute  for  Perception  TNO 

In  practice  sound  signals  to  be  recognized  and  classified 
are  usually  presented  against  a  background  of  noise.  The 
perceptual  uncertainty  introduced  by  the  noise  may  lead  to  the 
imagined  perception  of  signals  which  were  actually  not  present. 
This  will  be  explained  on  the  basis  of  experiments  on  the 
perceived  continuity  of  a  tone  alternated  with  noise,  triadic 
comparisons  of  signals  partly  masked  by  noise,  and  the 
subharmonic  pitches  of  a  pure  tone  at  low  S/N  ratio. 


CHAPTER  3:  The  Role  of  Stimulus  Uncertainty  in  the 
Discrimination  of  Auditory  Patterns 

Charles  S.  Watson  and  William  J.  Kelly 

Boys  Town  Institute  for  Communication  Disorders 
in  Children, 


A  review  of  earlier  research  on  the  discrimination  of  simple 
and  complex  sounds,  together  with  experiments  we  have  conducted 
with  tonal  sequences,  suggests  that  information  may  be  extracted 
from  word-length  auditory  patterns  by  application  of  highly 
efficient  mechanisms  of  selective  auditory  attention,  in  a 
two-stage  process.  A  descriptive  model  is  proposed,  according  to 
which  the  first  stage  in  the  processing  of  word-length  patterns 
is  assumed  to  be  a  crude  analysis  of  the  overall  pitch/prosody 
contour  of  the  pattern.  Processing  within  the  first  stage 
results  in  the  assignment  of  a  pattern  to  a  class  or  subset,  all 
of  the  members  of  which  carry  their  identifying  information  at 
the  same  spectral/temporal  points  within  the  pattern.  Processing 
within  the  second  stage  results  in  precise  resolution  of  the 
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pattern  waveform  in  the  high-information  bearing  region(s).  The 
primary  limitation  on  the  discr iminability  of  complex  auditory 
patterns  is  proposed  to  be  that  of  stimulus  uncertainty,  which 
has  its  major  effect  on  the  first  stage  of  the  processing  model. 


CHAPTER  4:  Meaningfulness  and  the  Perception  of 
Complex  Sounds 

John  C.  Webster 

Rochester  Institute  of  Technology 


In  recent  years,  computer-aided  statistical  decision 
techniques  based  on  electronic  and/or  visual  analyses  have  proven 
to  be  powerful  tools  for  classifying  passive  sonar  signals. 
Nonetheless,  they  have  not  equalled  the  capabilities  of  the  human 
auditory  system  in  classifying  these  complex  sounds.  This 
chapter  reviews  the  author's  experimental  work  over  a  number  of 
years  on  human  perception  of  complex  sounds  and  on  the  role  of 
meaningfulness  and  familiarity  in  signal  identification. 


CHAPTER  5:  Speech  Perception  and  Auditory  Processing 
A.  W.  F.  Huggins 
Bolt  Beranek  and  Newman  Inc. 


The  recognition  of  spoken  words  is  perhaps  our  most  highly 
developed  form  of  auditory  signal  classification.  A  large  part 
of  this  skill  is  dependent  on  the  top-down  processing  and 
recoding  of  the  input  that  is  made  possible  by  the  involvement  of 
language.  Other  contributing  factors  are  (1)  the  fact  that  the 
listener  himself  has  a  vocal  tract  that  is  functionally  identical 
to  the  one  that  produced  the  signal  he  must  classify;  and  (2)  the 
temporal  patterning  of  speech,  the  importance  of  which  has  only 
recently  been  appreciated.  Although  the  early  stages  of 
bottom-up  processing  of  speech  show  some  strong  similarities  to 
processing  of  non-speech  signals,  it  is  not  clear  to  what  extent 
useful  parallels  can  be  drawn  at  higher  levels.  This  uncertainty 
is  reflected  in  a  current  controversy  in  the  study  of  speech, 
summarized  in  the  question:  "Is  speech  special?" 
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SECTION  II:  PERCEPTION  OF  COMPLEX  VISUAL  PATTERNS 

CHAPTER  6:  Negligible  Symmetry  Effects  in  Dot  Pattern  Detection 
William  R.  Uttal  and  Thelma  E.  Tucker 
University  of  Michigan 


Both  information  and  autocorrelation  theories  predict  that 
symmetry  should  play  a  substantial  role  in  human  form  perception. 
Though  it  has  long  been  known  that  symmetry  effects  in  memory 
studies  are  substantial,  recent  evidence  suggests  that  they  are 
small  in  detection  tasks.  In  the  present  work,  it  is  further 
shown  that  symmetry  effects  in  a  task  involving  dotted  target 
detection  in  dotted  visual  noise  are  negligible.  This  finding 
adds  further  support  to  a  multistage  model  of  visual  form 
perception. 


CHAPTER  7:  A  Psychophysical  Approach  to  Dimensional  Integrality 

Robert  G.  Pachella,  Patricia  Somers, 
and  Mary  Hardzinski 

University  of  Michigan 


A  theory  of  dimensional  integrality  is  presented  in  which 
the  critical  defining  characteristic  of  integral  visual  displays 
is  the  lack  of  correspondence  between  the  physically  defined 
display  dimensions  and  the  attributes  of  the  display  that  can  be 
perceived  by  an  observer.  To  the  extent  that  the  physical 
dimensions  of  a  display  correspond  to  the  psychological 
attributes,  the  dimensions  will  be  separable;  to  the  extent  that 
they  don't  correspond,  the  dimensions  will  be  integral.  Two 
forms  of  data  are  presented.  First,  multidimensional  scaling 
data  are  presented,  which  define  these  complex  psychophysical 
mappings  for  various  sets  of  stimuli.  Second,  reaction  time 
data,  utilizing  these  scaled  stimuli,  are  presented  demonstrating 
how  this  psychophysical  correspondence  can  lead  to  patterns  of 
results  that  had  previously  been  associated  with  the  issues  of 
integrality  and  separability  of  display  dimensions. 
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SECTION  III:  THEORETICAL  APPROACHES  TO  PATTERN  RECOGNITION 

CHAPTER  8:  A  Feature  Extraction  Approach  to  Auditory  Pattern 
Recognition 

Julius  T.  Tou 

University  of  Florida 


Automatic  recognition  of  auditory  patterns  is  becoming  a 
subject  of  considerable  interest  among  psychologists.  This 
chapter  introduces  some  techniques  of  automatic  classification 
and  feature  extraction  which  are  applicable  to  auditory  patterns. 
Cluster-seeking  and  feature  extraction  approaches  are  discussed. 
The  Karhumen-Loeve  expansion  of  auditory  patterns  provides  a 
method  for  analysis  and  extraction  of  features.  Computer 
techniques  for  feature  extraction  are  discussed  in  which 
eigenvalues  are  used  to  represent  the  features.  It  is  found  that 
larger  eigenvalues  of  the  correlation  matrix  for  auditory  pattern 
vectors  carry  more  information  for  discrimination  of  the  auditory 
pattern. 


CHAPTER  9:  Pattern  Recognition  in  Ocean  Acoustics 
Arthur  E.  Bisson 

Naval  Ship  Research  and  Development  Center 


From  childhood  we  are  trained  to  recognize  and  identify 
out-of-sight  events  and  occurrences  by  means  of  only  the  acoustic 
noise  produced.  The  slam  of  a  car  door,  footsteps  in  the  hall, 
rain  outside  the  window  all  can  be  identified  by  distinguishing 
characteristics  or  patterns  that  differentiated  one  sound  or 
group  of  sounds  from  the  other.  How  does  man  recognize  or 
discriminate  one  sound  from  another?  What  are  the  basic  features 
or  characteristics  of  sounds  that  he  uses  to  discriminate? 
Although  questions  regarding  man's  learning  and  recognition 
process  are  not  easily  answered,  we  do  have  at  our  disposal  a 
formalism  that  aids  to  identify  and  utilize  distinct  features  of 
sounds  to  classify.  This  formalism  is  called  syntactic  pattern 
recognition.  This  chapter  is  devoted  to  an  overview  of  pattern 
recognition  as  it  applies  to  the  classification  of  acoustic 
sounds  in  the  ocean. 
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SECTION  IV:  MULTIDIMENSIONAL  PERCEPTUAL  SPACES 


CHAPTER  10:  Auditory  Perception:  Recommendations  for  a 
Computer  Assisted  Experimental  Paradigm 

Forrest  W.  Young 

University  of  North  Carolina 

and 

Cynthia  H.  Null 

College  of  William  and  Mary 


It  is  argued  that  understanding  the  perception  of  auditory 
stimuli  is  a  necessary  precursor  to  understanding  their 
recognitiion  and  classification.  Recommendations  for  a 
computer-assisted  experimental  paradigm  to  investigate  auditory 
perception  are  presented.  The  recommendations  focus  on  the  use 
of  digitized  stimuli,  multidimensioanl  scaling  and  multi-set 
matching.  The  design  should  permit  results  which:  (a)  emphasize 
individual  differences;  (b)  are  relatively  free  from 
experimenter  bias;  (c)  are  highly  replicable;  and  (d)  are 
validated . 


CHAPTER  11:  Multidimensional  Perception  Spaces;  Similarity 
Judgment  and  Identification 

David  J.  Getty,  John  A.  Swets  and  Joel  B.  Swets 
Bolt  Beranek  and  Newman  Inc. 


This  chapter  explores  the  relationship  between  the 
perceptual  representation  of  complex  stimuli  and  decision 
processes.  A  model  of  similar ity- judgment  and  identification 
processes  is  presented  that  assumes  (1)  that  complex  stimuli  are 
represented  perceptually  as  points  in  a  multidimensional 
geometric  space,  (2)  that  similarity  judgments  are  inversely 
related  to  inter  stimulus  distance  in  the  perceptual  space,  and 
(3)  that  identification  judgments  are  described  by  a 
probabilistic  decision  rule  based  on  the  pattern  of  interpoint 
distances  in  the  perceptual  space.  The  model  is  shown  to 
accurately  predict  individual  identification  confusion  matrices. 
The  results  suggest  that  observers'  identification  and  similarity 
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judgments  are  based  on  an  adaptive  weighting  of  the  perceptual 
dimensions.  This  optimization  process  is  both  stimulus  and  task 
dependent . 

CHAPTER  12:  Feature  Selection  in  Auditory  Perception 
James  H.  Howard,  Jr.  and  James  A.  Balias 
The  Catholic  University  of  America 


Feature  extraction  plays  an  important  role  in  theoretical 
accounts  of  auditory  classification.  Auditory  features  are 
generally  taken  to  represent  the  "essential"  perceptual  elements 
of  a  complex  sound.  The  feature  selection  problem  concerns  the 
definition  of  these  basic  elements.  Two  approaches  to  auditory 
feature  selection  are  briefly  reviewed.  The  results  of  three 
experiments  are  interpreted  as  supporting  the  more  flexible  of 
these  approaches. 
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The  Catholic  University  of  America 


Wednesday,  June  28 


INTRODUCTION 


DAVID  J.  GETTY 
JAMES  H.  HOWARD,  JR. 

PERCEPTION  OF  COMPLEX  AUDITORY  PATTERNS 

FREDERICK  L.  WIGHTMAN 
(Northwestern  University) 

"Pitch  Perception — An  Example  of  Auditory  Pattern 
Recognition. " 


REINIER  PLOMP 

(Institute  for  Perception  TNO) 

"Perception  of  Sound  Signals  under  Unfavorable  S/N  Ratios. 
CHARLES  S.  WATSON 

(Boys  Town  Institute  for  Communication  Disorders 
in  Children) 

"Physical  and  Psychological  Factors  in  the  Discrimination 
in  Word-Length  Auditory  Patterns." 

JOHN  C.  WEBSTER 

(Rochester  Institute  of  Technology) 

"Meaningfulness  and  the  Perception  of  Complex  Sounds." 

A.  W.  F.  HUGGINS 

(Bolt  Beranek  and  Newman  Inc.) 

"Speech  Perception  and  Auditory  Processing." 


Thursday,  June  29 

III.  PERCEPTION  OF  COMPLEX  VISUAL  PATTERNS 

WILLIAM  R.  UTTAL 
(University  of  Michigan) 

"Complexity  and  Symmetry  Effects:  Limitations  of  an 
Autocorrelation  Theory  of  Form  Detection." 

ROBERT  G.  PACHELLA,  PATRICIA  SOMERS,  and  MARY  HARDZ INSKI 
(University  of  Michigan) 

"A  Psychophysical  Approach  to  the  Problem  of  Dimensional 
Integrality." 


IV.  THEORETICAL  APPPROACHES  TO  PATTERN  RECOGNITION 
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